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1. Countries participating in PISA 2006 
but not members of the OECD in 2010 that 
had lower results than the United States 
include Croatia, Uruguay, Romania, Brazil, 
Argentina, Azerbaijan, Montenegro, Qatar, 
Tunisia, Colombia, Indonesia, Jordan and 
Kyrgyzstan. 



Executive Summary 

Maintaining our innovative edge in the world depends importantly on developing a 
highly qualified cadre of scientists and engineers. To realize that objective requires a 
system of schooling that produces students with advanced math and science skills. 

To see how well the U.S. as a whole, each state, and certain urban districts do at 
producing high- achieving math students, the percentage of U.S. public and private 
school students in the high- school graduating Class of 2009 who were highly accom- 
plished in mathematics in each of the 50 states and in 10 urban districts is compared 
to the percentages of similarly high achievers in 56 other countries. 

Unfortunately, the percentage of students in the U.S. Class of 2009 who were 
highly accomplished in math is well below that of most countries with which the U.S. 
generally compares itself. No less than 30 of the 56 other countries that participated 
in the Program for International Student Assessment (PISA) math test had a larger 
percentage of students who scored at the international equivalent of the advanced 
level on our National Assessment of Educational Progress (NAEP) tests. While 6 
percent of U.S. public and private school students rated as advanced in 8th-grade 
mathematics, 28 percent of Taiwanese students did. (See Figure 1, p. 16, for these 
results as well as for the relative rank internationally of each individual U.S. state.) 

It is not only Taiwan that did much, much better than the U.S. At least 20 percent 
of students in Hong Kong, Korea, and Finland were highly accomplished, and 12 other 
countries had at least twice the percentage of highly accomplished students as the 
U.S.: Switzerland, Belgium, the Netherlands, Liechtenstein, New Zealand, the Czech 
Republic, Japan, Canada, Macao, Australia, Germany, and Austria. The only members 
of the Organization for Economic Co-operation and Development (OECD) taking 
part in PISA 2006 that produced a smaller percentage of advanced math students than 
the U.S. were Spain, Italy, Israel, Portugal, Greece, Turkey, Chile and Mexico. The 
performance of the U.S. cannot be distinguished statistically from that of Russia. 1 

The percentage of students scoring at the advanced level varies considerably 
among the 50 states, but none does well in international comparison. Massachu- 
setts, with more than 1 1 percent advanced, does the best, but the performance 
of the Massachusetts Class of 2009 still trails that of 14 countries. Minnesota, 
ranked second among the 50 states, comes in at the same level as France, Swe- 
den, Denmark, Iceland, Slovenia and Estonia. California students are roughly 
comparable to those in Portugal, Italy, Israel and Turkey, and the lowest ranking 
states — West Virginia, New Mexico, and Mississippi — have a smaller percentage 
of high-performing students than do Serbia and Uruguay (although they do edge 
out Romania, Brazil, and Kyrgyzstan). 

In short, the percentages of high- achieving math students in the U.S. — and most 
of its individual states— are shockingly below those of many of the world’s leading 
industrialized nations. Results for many states are at the level of developing countries. 
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Executive Summary 



This is not simply the result of having a population that is heterogeneous and 
difficult to educate. Only 8 percent of white students in the U.S. Class of 2009 scored 
at the advanced level, a percentage that was less than the share of advanced students 
in 24 other countries regardless of their ethnic background. The percentage of white 
students in the state of New York rated as advanced (7.7) is roughly the same as 
the percentage of all students in Hungary and Norway; California’s white students, 
7.2 percent of whom score at the advanced level, are roughly even with all students 
in Poland and Ireland. 

The portion of students in the Class of 2009 with at least one parent who gradu- 
ated from college who are performing at the advanced level is 10.3 percent. In 16 
countries, students of all backgrounds, regardless of their parents’ education, do 
better than this advantaged segment of the U.S. population. The percentage of Illinois 
students with a college-educated parent who are highly accomplished is 9 percent, 
roughly the same percentage as for all students, regardless of background, in France 
and the U.K. Nearly 6 percent of Rhode Island’s students from college-educated 
backgrounds score at the advanced level, the same percentage as all students in Italy, 
Spain, and Latvia, regardless of background. Uruguay and Bulgaria produce the same 
proportion of advanced students, no matter their background, as found among chil- 
dren of the college-educated in Mississippi, just 2.2 percent. 

At the district level, while the percentages of highly accomplished public and 
private school students in Austin, Charlotte, and Boston exceed the U.S. as a whole, 
New York City trails these cities as well as Israel. San Diego, Houston, Washington, 

D. C., Chicago, Los Angeles, and Atlanta are all clustered below Uruguay and 
Bulgaria but above Chile, Thailand, Romania, Brazil, and Mexico, placing them 
at a level roughly equal to that of a Latin American country. 

Some have attributed this comparatively poor performance to the focus of the 
2002 federal accountability statute, No Child Left Behind (NCLB), on the educa- 
tional needs of very low performing students. But, in fact, the percentage of students 
performing at a high level in math climbed steadily in the years following the law’s 
passage. The incapacity of American schools to bring students up to the highest level 
of accomplishment in mathematics is much more deep-seated than anything induced 
by recent federal legislation. 

In sum, the U.S. trails other industrialized countries in bringing its students up to 
the highest levels of accomplishment in mathematics. It is not a story of some states’ 
high performance being offset by the low performance of other states. Nor is it a story 
of immigrant or disadvantaged or minority students hiding the good performance of 
better prepared students. Comparatively small percentages of white students in the states 
achieve at a high level. And only a small proportion of the children of our college-educated 
population is equipped to compete with students in a majority of OECD countries. ♦ 




Only 8 percent 
of white students in 
the U.S . Class of 2009 
scored at the 
advanced level. 
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Introduction 

“Although many people assume that the U.S. 
will always be a world leader in science and technology, 
this may not continue to be the case inasmuch as 
great minds and ideas exist throughout the world.” 

— Committee on Prospering in the Global Economy of the 21st Century (2005) 



1. Office of the Press Secretary, 

White House Office, “Remarks by the 
president on the “Educate to Innovate” 
Campaign and Science Teaching and 
Mentoring Awards,” January 6, 2010. 

2 . Goldin and Katz (2008). 

3 . Peterson (2010), Figures 1-5, pp. 268-272. 

4 . See Hanushek and Lindseth (2009), 
chapter 2. 



1 he economic and technological demand for a talented, well-educated, 
highly skilled population has never been greater. With rapidly advancing 
technologies in an increasingly integrated world economy, no one doubts the 
extraordinary importance of highly accomplished professionals. Not only must 
everyday workers have a set of technical skills surpassing those needed in the 
past, but a cadre of highly talented professionals trained to the highest level of 
accomplishment is needed to foster innovation and growth. In the words of 
President Barack Obama, “Whether it's improving our health or harnessing 
clean energy, protecting our security or succeeding in the global economy, our 
future depends on reaffirming America’s role as the world’s engine of scientific 
discovery and technological innovation. And that leadership tomorrow depends 
on how we educate our students today, especially in math, science, technology, 
and engineering.” 1 Unfortunately, the data show that our schools are not 
supporting levels of achievement that are competitive internationally. 

The U.S. has long recognized the importance of a well-educated work force. 
During the early decades of the 20th century, the country made disproportionately 
large investments in secondary and higher education, which translated into 
unparalleled growth that made the U.S. the dominant economic power in the world. 2 
But in recent years the performance of U.S. schools, once the envy of the world, 
has slipped, even as technological innovations have intensified the demand for 
human capital. The test-score performance of 17-year-old students on the National 
Assessment of Educational Progress (NAEP) has remained essentially unchanged for 
the past 40 years, and high school graduation rates have declined since the 1970s. 3 
While the U.S. has stagnated, other countries have advanced rapidly to emulate 
investments in human capital begun by the U.S. In fact, many other countries now 
exceed the U.S. in the provision of secondary and tertiary schooling, and, more 
importantly, they do dramatically better than the U.S. in terms of achievement. 4 
Those issues are the subject of this analysis. 
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The Demand for High Achievers 



How serious is the mismatch between the country’s needs for a highly 
skilled work force and the product of American schools? Is it just a few states or 
particular regions that are not producing as large a percentage of high achievers 
as other industrialized countries? Is the problem primarily the performance of 
students from minority racial and ethnic backgrounds or from homes where the 
parents lack a college education? Or do too many students, even white students 
or those from well-educated backgrounds, fail to achieve at a high level? These 
are some of the key questions that we address in this study. 



The Demand for High Achievers 

The gap between the burgeoning business demand for a highly accomplished 
workforce and a lagging educational system has steadily widened. Even as the U.S. 
was struggling with a near 10 percent unemployment rate in the summer of 2010, 
businesses complained that they could not find workers with needed skills. “The 
people that are out of work just don’t match the types of jobs that are here, open 
and growing,” says the head of a nonprofit group trying to make Cleveland a center 
for medical innovation. New York Times writer Motoko Rich says the complaints 
are not just coming from Ohio: “The problem.. .is a mismatch between the kind of 
skilled workers needed and the ranks of the unemployed.” 5 

Skill shortages have severe consequences for a nation’s overall productivity. 
Two of the authors of this report have shown elsewhere that countries with 
students who perform at higher levels in math and science show larger rates 
of increase in economic productivity than do otherwise similar countries 
with lower-performing students. 6 As Bill Gates, chairman of the Microsoft 
Corporation, has put it, “Unless the schools of the U.S. find the tools to bring 
students up to the highest level of accomplishment, it places the nation at risk in 
the international economy of the 21st Century. In particular, I’m concerned that 
too few young people are acquiring the knowledge they need to use technology in 
creative and innovative ways.” 7 

The public seems to have grasped the fact that American students are 
faltering in math and science relative to their peers in other countries. When the 
public was asked for the ranking of student performance in math as compared 
with other countries in the world, those interviewed were, on average, correct in 
identifying U.S. performance as below the median of all participating countries. 8 

Public discourse, however, has tended to focus on the need to address basic 
levels of achievement, particularly among disadvantaged students. This focus 
has been evident since the passage of the federal Elementary and Secondary 




“ Leadership tomorrow 
depends on how we 
educate our students 



today, especially in math, 
science, technology, 
and engineering 
— Barack Obama 



5. Rich (2010). 

6 . Hanushek and Woessmann (2008, 2009). 

7. Gates (2007). Lowell, Salzman, and 
Bernstein (2009) are unpersuaded that there 
is a crisis in STEM education, suggesting that 
the larger problem might be on the demand 
side. They do calculate, however, that a 
significantly smaller percentage of the most 
able U.S. high school students entered STEM 
higher education and subsequent STEM 
careers over the past two decades than did in 
the prior two decades. 

8 . Howell, Peterson, and West (2009), p. 27. 
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“Unless the schools of 
the U.S.find the tools to 
bring students up 
to the highest level of 
accomplishment, 
it places the nation at 
risk in the international 
economy of the 
21st Century.” 

—Bill Gates 



9 . Stem Education Coalition website, STEM 
Ed Coalition Objectives, accessed July 1, 

2010 at http://www.stemedcoalition.org/ 
content/ objectives/ 

10 . Committee on Prospering in the Global 
Economy of the 21st Century (2005). 

11 . Hanushek and Woessmann (2009). 

12. Bishop (1992); Murnane, Willett, and 
Levy (1995). 

13 . As quoted in Friedman (2007), p. 302. 

14 . As reported on Thinkport.org and 
quoted in Friedman (2007), p. 302. 



Education Act (ESEA) in 1965, now known in its most recent re-authorization 
as No Child Left Behind (NCLB). Both federal funding and the accountability 
elements of NCLB have stressed the importance of bringing every student up to a 
minimum level of proficiency. 

As great as this need may be, there is no less need to lift more students, 
no matter their socioeconomic background, to high levels of educational 
accomplishment. In 2006, the Science Technology Engineering and Math 
(STEM) Education Coalition was formed to “raise awareness in Congress, the 
Administration, and other organizations about the critical role that STEM 
education plays in enabling the U.S. to remain the economic and technological 
leader of the global marketplace for the 21st Century.” 9 In the words of 
a National Academy of Sciences report that jump-started the coalition’s 
formation, the nation needs to “increase” its “talent pool by improving K-12 
science and mathematics education.” 10 

In short, the U.S. cannot afford to neglect high performers in our quest 
to bring up the bottom. Performance at the top end is no less important, and 
improvements at both ends reinforce each other, helping to accelerate the growth 
in productivity of the nation’s economy. 11 



A Focus on Math 

To see how well U.S. schools do at producing high-achieving math students, 
we compare the percentage of U.S. public and private school students in the 
graduating Class of 2009 who were highly accomplished in mathematics in each 
of the 50 states and in 10 urban districts to percentages of high achievers in 56 
other countries. 

We give special attention to math performance because math appears to 
be the subject in which accomplishment in secondary school is particularly 
significant for both an individual’s and a country’s economic well-being. 
Existing research, though not conclusive, indicates that math skills better 
predict future earnings and other economic outcomes than other skills learned 
in high school. 12 “Choose math,” a Norwegian scholar has advised students, 
“because you will meet it more and more in the future. Math becomes more and 
more important in all areas of work and scholarship. There will be more math at 
work, so you will need more math at school.” 13 The American Diploma Project 
agrees with this assessment, estimating that “in 62 percent of American jobs 
over the next 10 years, entry-level workers will need to be proficient in algebra, 
geometry, data interpretation, probability and statistics.” 14 
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Data and Methodology 



If individuals can profit by investments in math education, the same is 
true for countries as a whole. In a prior study, two of the authors of this report 
demonstrate that growth in the economic productivity of a nation is driven 
more clearly by the math proficiency of its high school students than by their 
proficiency in other subjects. 15 

There is also a technical reason for focusing our analysis on math. This 
subject is particularly well suited to rigorous comparisons across countries and 
cultures. There is a fairly clear international consensus on the math concepts and 
techniques that need to be mastered and on the order in which those concepts 
should be introduced into the curriculum. The knowledge to be learned remains 
the same regardless of the dominant language spoken in a culture. Comparing 
reading performances is more challenging because of structural differences in 
languages, and science comparisons can be faulted for a lack of consensus on 
the science concepts that need to be mastered. (See Appendix A for a further 
discussion of U.S. reading and science performance in international perspective 
along with tables that provide the performance data of countries, states, and 
urban districts that are the focus of this analysis.) 



Data and Methodology 

Our analysis relies on test-score information from young adults collected by 
the National Assessment of Educational Progress (NAEP) and the Program 
for International Student Assessment (PISA). 16 NAEP, often called “the 
nation's report card,” is a large, nationally representative assessment of student 
performance in mathematics, reading, and science that has been administered 
periodically since the early 1970s to U.S. students in 4th grade and 8th grade, 
and at the age of seventeen. Since 2001, it has provided achievement data for a 
representative sample of students in each of the 50 states and a select number 
of urban school districts. PISA is an internationally standardized assessment of 
student performance in mathematics, science, and reading established by the 
Organization for Economic Co-operation and Development (OECD). It was 
administered in 2000, 2003, and 2006 to representative samples of 15-year-olds 
in all OECD countries as well as in many others. 17 

We focus on the performance of the international equivalent of the U.S. high 
school graduation Class of 2009 at the time when this class was in the equivalent 
of U.S. grades 8 and 9. The NAEP used was administered to 8th graders in 2005 
(NAEP 2005), while PISA 2006 was administered one year later to students at the 
age of 15, the year at which most Americans are in 9th grade. 



15 . Hanushek and Woessmann (2009), 
Table 2. 

16 . Data for NAEP come from the official 
website [accessed June 1, 2010], http://nces. 
ed.gov/nationsreportcard/. NAEP has also 
tested periodically a representative sample 
of students in several other subjects. Results 
from these other tests are not used here. 

17. The OECD which administers PISA 
is an international economic organization 
encompassing most of the high income, 
developed countries of the world. In 2007, 
it had 30 members; three new members 
(Chile, Israel, and Slovenia) were added 
in 2010. Information about PISA can 

be found in Organisation for Economic 
Co-operation and Development (2004), 
table 6.2, and Organisation for Economic 
Co-operation and Development (2007). 

Data for PISA 2006 come from the PISA 
microdata (http://www.pisa.oecd.org/). 

The PISA assessments build upon earlier 
international testing, most importantly 
those of the International Association for 
the Evaluation of Educational Achievement 
(IEA) now known as Trends in Mathematics 
and Science Survey (TIMSS). IEA has 
conducted assessments since the mid-1960s 
and is responsible for the TIMSS testing 
that is discussed below. See http://www. 
iea.nl/. Historical PISA scores and those of 
the Trends in International Mathematics 
and Science (TIMSS), used below, are 
summarized in Provasnik, Gonzales, and 
Miller (2009), which also contains references 
to the original publications for TIMSS. 
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Math appears to be 
the subject in which 
accomplishment 
in secondary school 
is particularly 
significant for both an 
individual’s and 
a country’s economic 
well-being. 



18 . Loveless (2008). 

19 . Mullis, Martin, and Foy (2008), p. 71. 

20 . Some have suggested that PISA and 
NAEP do not test a common domain of 
knowledge. Former Commissioner of the 
National Center for Education Statistics 
Mark Schneider objects to using PISA to 
test a representative sample of students in 
each state on the grounds that “PISA is a 
self-proclaimed yield study’ assessing the 
‘literacy’ of 15 -year- olds and is not tied to 
any specific curricula (Schneider (2009)).” 
Brookings scholar Tom Loveless (2009) has 
critiqued the validity of the PISA science 
test. But average student performances 
across countries on different international 
tests are strongly correlated, suggesting that 
a common domain of knowledge is being 
tested (Hanushek and Woessmann (2009)). 
For example, the correlation between TIMSS 
2007 and PISA 2007 was 0.93 (Phillips 
(2009), p. 36). 



NAEP is governed by the National Assessment Governing Board (NAGB), 
which consists of 26 educators and other public figures appointed by the U.S. 
Secretary of Education. In 2005, NAEP tested representative samples of 8th- 
grade public and private school students in each of the 50 states, in 10 large 
public school districts, and in the U.S. as a whole in math, science, and reading. 
For each of these jurisdictions, NAEP 2005 calculates the percentage of students 
who perform at three levels: basic, proficient, and advanced. The focus of this 
report is the top performers, the percentage of students NAEP found to perform 
at the advanced level. 

Only 6.04 percent of the students in the U.S. in 8th grade in 2005 scored at the 
advanced level in math. That the percentage is small is not by itself a definitive 
indication that only a few American 8th graders are highly accomplished in the 
subject. Since NAGB has the power to set the advanced bar at whatever level it 
deems appropriate, the specific level at which the standard is set is ultimately a 
matter of judgment by its board, which in turn is advised by experts in the field. 
Some critics feel that the standard set by the NAEP governing board is excessively 
stringent. 18 However, the 2007 Trends in International Math and Science Study 
(TIMSS 2007), another international test that has been administered to students 
throughout the world, appears to have set a standard very similar to NAEP 2005, as 
only 6 percent of U.S. 8th graders scored at the advanced level on that test as well. 19 

We do not take a position on this question but instead simply take the NAEP 

2005 standard as given and compare U.S. performance at that advanced level to 
that of other countries. We use information from the PISA 2006 mathematics 
test to estimate the percentage of students in other countries who would have 
scored at this same advanced level or higher had they taken NAEP 2005. We 
are able to estimate that percentage because students in the U.S. and in 56 other 
countries took the PISA 2006 math examination. (See Appendix Table A.2, p. 

30, for the list of countries who participated in the administration of the PISA 

2006 math examination.) Because U.S. students took both the NAEP 2005 and 
the PISA 2006, it is possible to find the score on the PISA that is tantamount to 
scoring at the advanced level on the NAEP, i.e., the score that will yield the same 
percentage of U.S. students as scored at the advanced level on the NAEP. 

A score on PISA 2006 of 617.1 points is equivalent to the lowest score 
obtained by anyone in the top 6.04 percent of U.S. students in the Class of 
2009. (The PISA assessment has an average score of 500 among OECD students 
and a standard deviation of 100.) It is assumed that both NAEP and PISA 
tests randomly select questions from a common universe of mathematics 
knowledge. 20 Given that assumption, it may be further assumed that students 
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United States Advanced Math Performance in World Perspective 



who scored similarly on the two exams will have similar math knowledge, i.e., 
students who scored 617.1 points or better on the PISA test would have been 
identified as advanced had they taken the NAEP math test. Inasmuch as a score 
of 617.1 points is more than one standard deviation above the average student 
score on the PISA, it is clear that a group of highly accomplished students has 
been isolated. (For more methodological details, see Appendix B.) 

As stated above, NAEP examinations are given to 8th graders, while PISA 
examinations are given at the age of 15, the age of the average U.S. 9th grader, 
so tracking the Class of 2009 means relying on the 2005 NAEP test and the PISA 
test of 2006. 21 In comparing the performance of the Class of 2009 on the NAEP 
and PISA tests at these two different points in time, we assume that no event 
happened between 8th and 9th grade that significantly altered the performance 
of American students relative to that of students in other countries. 22 

Because representative samples of student performance on the NAEP 2005 
are available for each state and for 10 urban school districts, it is possible to 
compare the percentages of students in the Class of 2009 who scored at the 
advanced level for each state and for 10 urban districts to the percentage of 
equally advanced students in countries from around the globe. 

In short, linking the scores of the Class of 2009 on NAEP 2005 and PISA 2006 
provides us with the opportunity to assess from an international vantage point 
how well the U.S. as a whole, individual states, and certain school districts are 
doing at lifting students to high levels of accomplishment. 

United States Advanced Math 
Performance in World Perspective 

We first provide an overall assessment of the relative percentages of adolescents 
in the U.S. and other countries who have reached a very high level of 
mathematics achievement. Largely as a way of explaining away the disappointing 
relative performance of U.S. students, it is frequently noted that the U.S. has a 
very heterogeneous population with large numbers of immigrants. Such a diverse 
population, with students coming to school with varying preparation, may 
handicap U.S. performance relative to other, more homogeneous countries. For 
this reason, we provide two additional analyses. We examine two U.S. subgroups 
conventionally thought to have better preparation for school— white students 
and students from families where at least one parent is reported to have received 
a college degree— and compare the percentages of high- achieving students 
among them to the (total) populations abroad. 




No less than 30 of the 
56 other countries that 
participated in the PISA 
math test had a larger 
percentage of students 
who scored at 
the international 
equivalent of the 
advanced level 



21. It is fortunate that the NAEP math, 
science and reading tests were given in 2005 
and the PISA math, science and reading 
tests were given in 2006, as those are the 
only years in the 21st century when that 
coincidence occurred. 

22. A similar analysis could be made 
using the 2007 Trends in International 
Mathematics and Science Study (TIMSS 
2007), which also administered a 
mathematics examination to a representative 
sample of 8th grade students in the United 
States and 49 other countries around the 
world (Mullis, Martin, and Foy (2008)). 
However, as is discussed further below, 
TIMSS 2007 was not administered to 
students in many industrialized (OECD) 
countries that out- scored the United States 
on the PISA. 



U.S. MATH PERFORMANCE IN GLOBAL PERSPECTIVE 11 



United States Advanced Math Performance in World Perspective 



23 . Other countries participating in PISA 
2006 but not current members of the OECD 
who had lower results than the US include 
Russia (although not significantly so), Croatia, 
Uruguay, Romania, Brazil, Argentina, 
Azerbaijan, Montenegro, Qatar, Tunisia, 
Colombia, Indonesia, Jordan and Kyrgyzstan. 



Overall Results 

The percentage of public and private school students in the U.S. Class of 2009 
who were highly accomplished is well below that of most countries with which 
the U.S. generally compares itself. No less than 30 of the 56 other countries 
that participated in the PISA math test had a larger percentage of students who 
scored at the international equivalent of the advanced level. While just 6 percent 
of U.S. students earned at least 617.1 points on the PISA 2006 exam, 28 percent 
of Taiwanese students did. (See Figure 1, p. 16, for these results as well as for the 
relative rank internationally of each individual U.S. state.) 

It is not only Taiwan that did dramatically better than the U.S. At least 
20 percent of students in Hong Kong, Korea, and Finland were also highly 
accomplished. Twelve other countries had more than twice the percentage of 
highly accomplished students as the U.S.: In order of math excellence, they are 
Switzerland, Belgium, the Netherlands, Liechtenstein, New Zealand, the Czech 
Republic, Japan, Canada, Macao, Australia, Germany, and Austria. 

The remaining countries that educate to a high level of accomplishment a 
higher proportion of their students than the U.S. are Slovenia, Denmark, Iceland, 
France, Estonia, Sweden, the U.K., the Slovakia, Luxembourg, Hungary, Poland, 
Norway, Ireland and Lithuania. 

This 30-country list includes virtually all the advanced industrialized 
countries of the world, most of whom are members of the OECD. The only 
countries currently members of the OECD countries that produce a smaller 
percentage of advanced math students than the U.S. are Spain, Italy, Israel, 
Portugal, Greece, Turkey, Chile and Mexico. 23 Additionally, 24 non-OECD 
countries participated in PISA 2006. 

The percentage of students scoring at the advanced level varies among the 
50 states. Massachusetts, with more than 1 1 percent advanced, does better than 
any other state, but the percentage of students in the Massachusetts Class of 
2009 showing advanced skills trails those of 14 countries. Minnesota ranked 
second among the 50 states; its level of performance is roughly equal to that of 
France, Sweden, and Denmark. See Table 1 for a comparison of all states with 
performances abroad. 

Even though California is known for its Silicon Valley, just 4.5 percent of the 
students in the Silicon Valley state are performing at a high level, a percentage 
roughly comparable to that of Portugal. The lowest ranking states— West 
Virginia, New Mexico, and Mississippi — have a smaller percentage of the 
highest-performing students than do Serbia and Uruguay, although they edge 
out Romania, Brazil, and Kyrgyzstan. 
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Table 1 



Percentages of all students at the advanced level per state and countries with similar and higher percentages 
at the advanced level in overall student population 



State 


Percent 

advanced 


Significantly 
outperformed by* 


Countries with similar percentages 
of advanced students** 


1. 


Massachusetts 


11.4% 


14 


Austria • Germany • Denmark • France • Iceland • Slovenia 


2. 


Minnesota 


10.8 


16 


Denmark • Estonia • France • Iceland • Slovenia • Sweden 


3. 


Vermont 


8.8 


22 


U.K. • Hungary • Ireland • Luxembourg • Norway • Poland • Slovakia 


4. 


New Jersey 


8.7 


18 


Estonia • France • U.K. • Hungary • Ireland • Iceland • Lithuania • Luxembourg • Norway • Poland • Slovakia • Sweden 


4. 


Washington 


8.7 


21 


U.K. • Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Slovakia • Sweden 


6. 


Virginia 


7.9 


22 


U.K. • Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Slovakia 


7. 


Connecticut 


7.8 


23 


Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Slovakia 


8. 


Oregon 


7.3 


25 


Hungary • Ireland • Lithuania • Poland 


9. 


North Carolina 


7.1 


27 


Slovakia • Ireland • Lithuania • Poland 


10. 


Maryland 


6.8 


29 


Lithuania • Russia 


11. 


South Carolina 


6.7 


29 


Lithuania • Russia 


11. 


Wisconsin 


6.7 


29 


Lithuania • Russia 


13. 


Ohio 


6.6 


29 


Lithuania • Russia 


14. 


New Hampshire 


6.5 


29 


Lithuania • Russia 


14. 


South Dakota 


6.5 


29 


Lithuania • Russia 


16. 


Colorado 


6.3 


29 


Spain • Lithuania • Russia 


16. 


New York 


6.3 


29 


Lithuania • Russia 


18. 


Texas 


6.2 


29 


Lithuania • Russia 




United States 


6.0 


30 


Russia 


19. 


Nebraska 


6.0 


29 


Spain • Lithuania • Russia 


20. 


Alaska 


5.8 


30 


Spain • Latvia • Russia 


21. 


Iowa 


5.7 


30 


Spain • Latvia • Russia 


21. 


Pennsylvania 


5.7 


30 


Spain • Israel • Italy • Lithuania • Latvia • Russia 


23. 


Montana 


5.6 


30 


Spain • Latvia • Russia 


24. 


Michigan 


5.5 


30 


Spain • Israel • Italy • Latvia • Russia 


25. 


Illinois 


5.4 


30 


Spain • Israel • Italy • Latvia • Russia 


26. 


Kansas 


5.2 


30 


Spain • Israel • Italy • Latvia • Russia 


27. 


Indiana 


5.1 


30 


Spain • Israel • Italy • Latvia • Russia 


28. 


Delaware 


5.0 


32 


Israel • Italy • Latvia 


28. 


Maine 


5.0 


32 


Israel • Italy • Latvia 


30. 


North Dakota 


4.8 


32 


Israel • Italy • Latvia • Portugal • Turkey 


31. 


Utah 


4.7 


32 


Israel • Italy • Latvia • Portugal • Turkey 


32. 


Arizona 


4.6 


33 


Israel • Italy • Portugal • Turkey 


32. 


Florida 


4.6 


32 


Israel • Italy • Latvia • Portugal • Turkey 


34. 


California 


4.5 


33 


Israel • Italy • Portugal • Turkey 


34. 


Idaho 


4.5 


33 


Israel • Italy • Portugal • Turkey 


36. 


Georgia 


4.3 


34 


Greece • Israel • Portugal • Turkey 


37. 


Missouri 


4.1 


35 


Greece • Portugal • Turkey 


38. 


Wyoming 


3.5 


37 


Croatia • Turkey 


39. 


Kentucky 


3.4 


37 


Croatia • Turkey 


40. 


Rhode Island 


3.3 


37 


Croatia • Turkey 


41. 


Nevada 


3.1 


38 


Turkey 


42. 


Arkansas 


3.0 


38 


Bulgaria • Turkey 


43. 


Tennessee 


2.9 


38 


Bulgaria • Turkey 


44. 


Hawaii 


2.5 


38 


Bulgaria • Turkey • Uruguay 


45. 


Oklahoma 


2.4 


38 


Bulgaria • Serbia • Turkey • Uruguay 


46. 


Alabama 


2.3 


38 


Bulgaria • Serbia • Turkey • Uruguay 


47. 


Louisiana 


1.7 


42 


Bulgaria 


48. 


West Virginia 


1.4 


42 


Bulgaria 


48. 


New Mexico 


1.4 


42 


Bulgaria 


50. 


Mississippi 


1.3 


42 


Turkey 



*Number of countries whose percent advanced was statistically significantly higher 

**Countries where the percentage of students at the advanced level did not differ significantly from state. If no country had similar percentage, the country with the percentage just higher than that of the state is listed. 



U.S. MATH PERFORMANCE IN GLOBAL PERSPECTIVE 13 





United States Advanced Math Performance in World Perspective 



In short, the percentages of high-achieving students in the U.S.— and in 
most of its individual states — are shockingly below those of many of the world’s 
leading industrialized nations. Results for many states are at a level equal to those 
of developing countries. 

White Students 

The overall news is sobering. Some might try to comfort themselves by saying the 
problem is limited to large numbers of students from immigrant families, or to 
African American students and others who have suffered from discrimination. 
For example, the statement by the STEM Coalition that we “encourage more 
of our best and brightest students, especially those from underrepresented or 
disadvantaged groups, to study in STEM fields” suggests that the challenges are 
concentrated in non-white segments of the U.S. population. 

Without denying that the paucity of high-achieving students within 
minority populations is a serious issue, let us consider the other side of that 
coin and inquire about the performance of white students for whom the case of 
discrimination cannot easily be made. Figure 2, p. 16, compares the percentage 
of U.S. white students in the Class of 2009 who scored at the advanced level 
with the percentage of all students in other countries. Note that in this figure no 
adjustment is made in any other participating country for the size of its minority 
population. U.S. white students are being compared to all students, of whatever 
ethnic or racial background, in the other countries. If the issue of math education 
is strictly a minority group issue, then this chart can be expected to show the U.S. 
as one of the world leaders. 

Figure 2 reveals that to be far from the case. In 24 countries, the percentage 
of highly accomplished students (from all ethnic backgrounds) surpasses 
that in the U.S. white student population in the Class of 2009, 8 percent of 
whom score at the advanced level. The percentage of white students at the 
advanced level in the state of New York was 7.7 percent, roughly the same 
as the percentage of all students in Hungary and Poland. In California, 7.2 
percent of white students are performing at the advanced level, a percentage 
insignificantly different from the percentage of all students in Ireland and 
Lithuania. Table 2 provides a full comparison of white students in each state 
with the performances of all students abroad. 
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Table 2 



Percentages of white students at the advanced level per state and countries with similar and higher percentages at the 
advanced level in overall student population 



Percent Significantly Countries with similar percentages 



State 


advanced 


outperformed by 


* of advanced students** 


1. Massachusetts 


12.6% 


11 


Australia • Austria • Germany • Denmark • Liechtenstein • Macao • Slovenia 


2. Minnesota 


12.3 


12 


Australia • Austria • Germany • Denmark • Liechtenstein • Slovenia 


3. New Jersey 


11.5 


12 


Australia • Austria • Germany • Denmark • Estonia • France • Iceland • Liechtenstein • Slovakia • Slovenia • Sweden 


4. South Carolina 


10.8 


15 


Austria • Denmark • Estonia • France • Iceland • Slovakia • Slovenia • Sweden 


5. Connecticut 


10.6 


16 


Denmark • Estonia • France • Iceland • Slovakia • Slovenia • Sweden 


5. Texas 


10.6 


16 


Denmark • Estonia • France • Iceland • Slovenia • Sweden 


7. Virginia 


10.3 


15 


Austria • Denmark • Estonia • France • U.K. • Hungary • Iceland • Luxembourg • Poland • Slovakia • Slovenia • Sweden 


8. North Carolina 


9.9 


16 


Denmark • Estonia • France • U.K. • Hungary • Iceland • Luxembourg • Norway • Poland • Slovakia • Slovenia • Sweden 


9. Maryland 


9.8 


16 


Denmark • Estonia • France • U.K. • Hungary • Iceland • Luxembourg • Norway • Poland • Slovakia • Slovenia • Sweden 


10. Washington 


9.5 


17 


Denmark • Estonia • France • U.K. • Hungary • Iceland • Luxembourg • Norway • Poland • Slovakia • Sweden 


11. Vermont 


9.0 


19 


Estonia • France • U.K. • Hungary • Luxembourg • Norway • Poland • Slovakia • Sweden 


12. Alaska 


8.5 


21 


U.K. • Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Slovakia • Sweden 


13. Colorado 


8.4 


19 


Estonia • France • U.K. • Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Slovakia • Sweden 


United States 


8.0 


24 


Poland • Slovakia • Norway • Lithuania • Ireland • Hungary 


14. Oregon 


7.9 


22 


U.K. • Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Slovakia 


15. New York 


7.7 


23 


Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Slovakia 


16. Ohio 


7.6 


24 


Hungary • Ireland • Lithuania • Norway • Poland • Slovakia 


16. Wisconsin 


7.6 


23 


Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Slovakia 


16. Arizona 


7.6 


23 


Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Slovakia 


16. Illinois 


7.6 


23 


Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Slovakia 


20. South Dakota 


7.3 


25 


Hungary • Ireland • Lithuania • Poland 


21. California 


7.2 


27 


Slovakia • Ireland • Lithuania • Poland 


22. Florida 


7.1 


24 


Hungary • Ireland • Lithuania • Norway • Poland • Russia • Slovakia 


22. Delaware 


7.1 


26 


Hungary • Ireland • Lithuania • Poland • Russia 


24. Nebraska 


6.8 


29 


Lithuania • Russia 


24. Michigan 


6.8 


24 


Spain • Hungary • Ireland • Lithuania • Latvia • Norway •Poland • Russia • Slovakia 


26. New Hampshire 


6.5 


29 


Lithuania • Russia 


27. Pennsylvania 


6.4 


29 


Spain • Lithuania • Russia 


28. Georgia 


6.3 


29 


Spain • Lithuania • Latvia • Russia 


28. Kansas 


6.3 


29 


Spain • Lithuania • Russia 


30. Iowa 


6.2 


29 


Spain • Lithuania • Russia 


31. Montana 


6.0 


29 


Spain • Lithuania • Latvia • Russia 


32. Indiana 


5.9 


29 


Spain • Lithuania • Latvia • Russia 


33. Utah 


5.4 


30 


Spain • Israel • Italy • Latvia • Russia 


34. North Dakota 


5.3 


30 


Spain • Israel • Italy • Latvia • Russia 


35. Maine 


5.1 


32 


Israel • Italy • Latvia 


36. Idaho 


5.0 


32 


Israel • Italy • Latvia • Turkey 


36. Missouri 


5.0 


31 


Spain • Israel • Italy • Latvia • Portugal • Turkey 


38. Nevada 


4.6 


32 


Greece • Israel • Italy • Latvia • Portugal • Turkey 


39. Rhode Island 


4.2 


35 


Greece • Portugal • Turkey 


40. Wyoming 


3.8 


36 


Greece • Croatia • Turkey 


40. Arkansas 


3.8 


36 


U.K. • Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Slovakia • Sweden 


42. Tennessee 


3.7 


36 


Greece • Croatia • Turkey 


43. Kentucky 


3.6 


36 


Greece • Croatia • Turkey 


43. Alabama 


3.6 


34 


Bulgaria • Greece • Croatia • Israel • Portugal • Turkey • Uruguay 


45. Hawaii 


3.4 


32 


Bulgaria • Greece • Croatia • Israel • Italy • Latvia • Portugal • Serbia • Turkey • Uruguay 


46. New Mexico 


3.2 


36 


Bulgaria • Greece • Croatia • Turkey 


47. Oklahoma 


3.0 


38 


Bulgaria • Turkey 


48. Louisiana 


2.6 


38 


Bulgaria • Serbia • Turkey • Uruguay 


49. Mississippi 


2.4 


38 


Bulgaria • Serbia • Turkey • Uruguay 


50. West Virginia 


1.3 


42 


Turkey 



* Number of countries whose percent advanced was statistically significantly higher 

** Countries where the percentage of overall students advanced did not differ significantly from the percentage of advanced white students in the state. If no country had similar percent- 
age, the country with the percentage just higher than that of the state is listed. 
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Class of 2009: Percentage of students at advanced level in math in U.S. states and 
countries participating in PISA 2006. (Figure 1) 
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Note: Excludes participating countries below 1 percent: Romania, Brazil, Argentina, Azerbaijan, Mexico, Montenegro, Qatar, Tunisia, Columbia, Indonesia, Jordan, 
and Kyrgyzstan. 



24. This is only an estimate of parental 
education, as students tend to over- report 
parental attainment. In a study of high 
school sophomores, where parents reported 
their own education, 38 percent reported 
that at least one parent had a college diploma 
(Education Longitudinal Study of 2002). 



Children of Parents with a College Degree 

Another possibility is that schools help students reach levels of high 
accomplishment if parents are providing the necessary support. To explore this 
possibility, we assumed that students who reported that at least one parent had 
graduated from college were most likely to be given the kind of support that is 
needed to reach high levels of achievement. Approximately 45 percent of all U.S. 
students reported that at least one parent had a college degree. 24 

When we compared these U.S. students from highly educated families to all 
students in other countries, without regard to their parents’ education, we expected 
to find that the U.S. would place among the world leaders. But as can be seen in 
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Class of 2009: Percentage of white students in U.S. states at advanced level in math and 
percentage of all students at that level in countries participating in PISA 2006. (Figure 2) 
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Note: See note to Figure 1. 



Figure 3, p. 16, the percentage of students in the Class of 2009 whose parent had 
graduated from college and who are performing at the advanced level is just 10.3 
percent of the total. Students in 16 countries, no matter their parents' educational 
attainment, out-rank this more-advantaged segment of the U.S. population. 

The percentage of Illinois students from college-educated families 
who are highly accomplished, 9 percent, is similar to the percentage of 
all students in France and Great Britain. Rhode Island's students from 
college-educated backgrounds, 6 percent of whom are advanced, are doing 
no better than all the students in Italy and Spain. Uruguay and Bulgaria 
produce a similar proportion of advanced students — about 2 percent — as is 
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Class of 2009: Percentage of students with at least a college-educated parent in U.S. states 
at advanced level in math and percentage of all students at that level in countries participating 
in PISA 2006. (Figure 3) 
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Note: See note to Figure 1. 



found among children of the college-educated in Mississippi. Table 3, p.20, 
provides the percentage of students from college-educated backgrounds 
who perform at the advanced level in each state in comparison with all 
students in countries abroad. 

Urban School Districts 

Given the comparatively low performance of students from families where a parent 
has a college degree, it is not surprising to learn that the percentage of high-achieving 
students attending schools in urban school districts generally trails that in other 
countries by an especially wide margin. In Figure 4, that information is displayed in the 



18 



educationnext.org 



hks.harvard.edu/pepg 



United States Advanced Math Performance in World Perspective 



Class of 2009: Percentage of at advanced level in math in U.S. urban districts and 
countries participating in PISA 2006. (Figure 4) 
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Note: See note to Figure 1. 



same way as the information for states was displayed in the three preceding figures. 

The percentages of highly accomplished students in the three 
university cities of Austin (Texas), Charlotte (North Carolina), and Boston 
(Massachusetts) are higher than found in the U.S. as a whole. New York 
City trails Israel but slightly outperforms Portugal. San Diego, Houston, 
Washington, D. C., Chicago, Los Angeles, and Atlanta are all clustered below 
Uruguay and Bulgaria but above Chile, Thailand, Romania, Brazil, and Mexico. 
In other words, the ability of the schools in these districts to lift student 
performance to the highest level is roughly equal to that of schools in a Latin 
American country. 
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Table 3 



Percentages of advanced students with a college educated parent per state and countries with similar 
and higher percentages advanced in overall student population 



Percent Significantly Countries with similar percentages 

State advanced outperformed by* of advanced students 



1. Massachusetts 


17.1% 


4 


Belgium • Canada • Switzerland • Czech Rep • Japan • Liechtenstein • Netherlands • New Zealand 


2. Minnesota 


15.7 


5 


Australia • Austria • Canada • Switzerland • Czech Rep • Germany • Japan • Liechtenstein • Macao • Netherlands • New Zealand 


3. Washington 


14.1 


7 


Australia • Austria • Canada • Czech Rep • Germany • Japan • Liechtenstein • Macao • New Zealand • Slovenia 


4. Vermont 


14.0 


7 


Australia • Austria • Canada • Czech Rep • Germany • Japan • Liechtenstein • Macao • New Zealand 


5. Virginia 


13.0 


7 


Australia • Austria • Canada • Czech Rep • Germany • Denmark • Japan • Liechtenstein • Macao • New Zealand • Slovenia 


6. New Jersey 


12.9 


7 


Australia • Austria • Canada • Czech Rep • Germany • Denmark* Japan • Liechtenstein • Macao • New Zealand • Slovenia 


7. Oregon 


12.7 


8 


Australia • Austria • Canada • Czech Rep • Germany • Denmark • Japan • Liechtenstein • Macao • Slovenia 


8. Connecticut 


12.5 


11 


Australia • Austria • Germany • Denmark • Liechtenstein • Macao • Slovenia 


8. North Carolina 


12.5 


7 


Australia • Austria • Canada • Czech Rep • Germany • Denmark • Estonia • France • Iceland • Japan • Liechtenstein • Macao 
New Zealand • Slovakia • Slovenia • Sweden 


10. Texas 


11.7 


14 


Austria • Germany • Denmark • Slovenia 


11. Maryland 


11.6 


12 


Australia • Austria • Germany • Denmark • Estonia • France • Iceland • Liechtenstein • Slovenia • Sweden 


12. South Carolina 


11.3 


14 


Austria • Germany • Denmark • Estonia • France • Iceland • Slovakia • Slovenia • Sweden 


13. Ohio 


11.0 


14 


Austria • Germany • Denmark • Estonia • France • Iceland • Slovakia • Slovenia • Sweden 


13. Colorado 


11.0 


12 


Australia • Austria • Germany • Denmark • Estonia • France • U.K. • Hungary • Iceland • Liechtenstein • Poland • Slovakia 
Slovenia • Sweden 


15. Wisconsin 


10.8 


14 


Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Slovakia 


United States 


10.3 


17 


Denmark • Estonia • France • Iceland • Sweden 


16. New Hampshire 


10.2 


16 


Denmark • Estonia • France • U.K. • Hungary • Iceland • Luxembourg • Poland • Slovakia • Slovenia • Sweden 


17. New York 


10.0 


17 


Denmark • Estonia • France • U.K. • Hungary • Iceland • Poland • Slovakia • Sweden 


18. Arizona 


9.6 


17 


Denmark • Estonia • France • U.K. • Hungary • Iceland • Luxembourg • Norway • Poland • Slovakia • Sweden 


18. Pennsylvania 


9.6 


16 


Denmark • Estonia • France • U.K. • Hungary • Ireland • Iceland • Luxembourg • Norway • Poland • Slovakia • Slovenia • Sweden 


20. California 


9.5 


17 


Denmark • Estonia • France • U.K. • Hungary • Iceland • Luxembourg • Poland • Slovakia • Sweden 


21. South Dakota 


9.3 


17 


Denmark • Estonia • France • U.K. • Hungary • Iceland • Luxembourg • Norway • Poland • Slovakia • Sweden 


22. Indiana 


9.1 


17 


Denmark • Estonia • France • U.K. • Hungary • Ireland • Iceland • Lithuania • Luxembourg • Norway • Poland • Slovakia • Sweden 


22. Illinois 


9.1 


17 


Denmark • Estonia • France • U.K. • Hungary • Ireland • Iceland • Lithuania • Luxembourg • Norway • Poland • Slovakia • Sweden 


24. Nebraska 


9.0 


18 


Estonia • France • U.K. • Hungary • Iceland • Luxembourg • Norway • Poland • Slovakia • Sweden 


24. Iowa 


9.0 


18 


Estonia • France • U.K. • Hungary • Ireland • Iceland • Lithuania • Luxembourg • Norway • Poland • Slovakia • Sweden 


26. Kansas 


8.8 


18 


Estonia • France • U.K. • Hungary • Ireland • Iceland • Lithuania • Luxembourg • Norway • Poland • Slovakia • Sweden 


27. Montana 


8.5 


22 


U.K. • Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Slovakia 


27. Michigan 


8.5 


17 


Denmark • Estonia • France • U.K. • Hungary • Ireland • Iceland • Lithuania • Luxembourg • Norway • Poland • Russia 
Slovakia • Sweden 


29. Delaware 


8.1 


22 


U.K. • Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Slovakia 


30. Florida 


8.0 


22 


U.K. • Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Slovakia 


31. Georgia 


7.9 


22 


U.K. • Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Slovakia 


32. Utah 


7.5 


24 


Hungary • Ireland • Lithuania • Norway • Poland • Slovakia 


33. Maine 


7.4 


24 


Hungary • Ireland • Lithuania • Norway • Poland • Slovakia 


34. Idaho 


7.3 


23 


Hungary • Ireland • Lithuania • Luxembourg • Norway • Poland • Russia • Slovakia 


35. Missouri 


6.9 


17 


Denmark • Estonia • France • U.K. • Hungary • Ireland • Iceland • Lithuania • Luxembourg • Norway • Poland • Russia 
Slovakia • Sweden 


36. Kentucky 


6.6 


25 


Spain • Hungary • Ireland • Lithuania • Latvia • Poland • Russia • Slovakia 


37. North Dakota 


6.4 


29 


Spain • Lithuania • Latvia • Russia 


38. Nevada 


6.2 


27 


Spain • Ireland • Israel • Italy • Lithuania • Latvia • Poland • Portugal • Russia • Turkey 


39. Tennessee 


5.8 


29 


Spain • Israel • Italy • Lithuania • Latvia • Russia 


39. Rhode Island 


5.8 


29 


Spain • Israel • Italy • Lithuania • Latvia • Russia 


41. Arkansas 


5.4 


30 


Spain • Israel • Italy • Latvia 


42. Wyoming 


5.3 


30 


Russia • Spain • Israel • Italy • Latvia • Portugal • Russia • Turkey 


43. Alabama 


4.6 


29 


Bulgaria • Spain • Greece • Croatia • Israel • Italy • Lithuania • Latvia • Portugal • Russia • Turkey 


44. Oklahoma 


4.3 


33 


Greece • Israel • Italy • Portugal 


45. Hawaii 


4.2 


33 


Turkey • Greece • Croatia • Israel • Italy • Portugal • Turkey 


46. West Virginia 


3.0 


37 


Bulgaria • Croatia • Turkey 


46. Louisiana 


3.0 


35 


Bulgaria • Greece • Croatia • Portugal • Serbia • Turkey • Uruguay 


48. New Mexico 


2.9 


37 


Bulgaria • Croatia • Turkey • Uruguay 


49. Mississippi 


2.2 


38 


Bulgaria • Serbia • Turkey • Uruguay 



* Number of countries whose percent advanced was statistically significantly higher 
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Did No Child Left Behind Shift the 

Focus Away from the Best and the Brightest? 

Some attribute the comparatively small percentages of students performing at the 
advanced level to the focus of the 2002 federal accountability statute, No Child 
Left Behind (NCLB), on the educational needs of very low performing students. 25 
That law mandates that every student be brought up to the level a state deems 
proficient, a standard that most states set well below the NAEP standard of full 
proficiency, to say nothing of the advanced level that is the focus of this report. 

In order to comply with the federal law, some assert, schools are concentrating 
all available resources on the educationally deprived, leaving advanced students 
to fend for themselves. If so, then we should see a decline in the percentage of 
students performing at NAEP’s advanced level subsequent to the passage of the 
2002 federal law. In mathematics, however, the opposite has happened. As can 
be seen in Figure 5, the percentage performing at the advanced level was only 3.7 
percent in 1996 and 4.7 percent in the year 2000. But the percentage performing at 
that level subsequently climbed to 7.9 percent by 2009. If one assumes that NCLB 
did not have an impact on schools until after 2003, the increment in the percentage 
advanced is from 5.4 percent in that year to 7.9 percent in 2009. 26 



Percentage of 8th grade students at the advanced level and below 
basic level in mathematics on National Assessment of Educational 
Progress, 1996 to 2009. (Figure 5) 



Percent At Advanced Level 




Percentage 




The incapacity 
of American schools 
to bring students 
up to the highest level of 
accomplishment 

in mathematics 
is much more 
deep-seated than 
anything induced 
by recent federal 
legislation 



25. Loveless (2008). 

26. Education historian Diane Ravitch, 
among others, has objected to identifying 
NCLB effects as early as one year after the 
law was passed; see Ravitch (2010), p. 109. 
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27. Data available from authors upon 
request. Tom Loveless has reached quite 
different conclusions from an examination of 
this same information (Loveless, 2008). His 
findings depend upon his assumption that 
NCLB was influencing school policy by 2000, 
two years before the law was enacted. Apart 
from the problems with this assumption, any 
conclusions that are sensitive to the choice of 
one or another year near the cusp are hardly 
robust; also his analysis extends only to 2007 
and high achievers showed a growth spurt 
between then and 2009. 

28. Dee and Jacob (2009) find no impact of 
NCLB on NAEP reading performance. 

29. Phillips (2007, 2009). For other studies 
that compare test-score performances 
across countries, see Hanushek and 
Woessmann (2010). 



It is true that the percentage performing below the basic level decreased, by 
6.8 percentage points between 2000 and 2009, from 34.2 percent to 27.4 percent 
(see Figure 5, p.21), but that is only a 20 percent change, as compared to the 69 
percent change in the percentage advanced over the same period. One should not 
put any particular weight on percent changes in percentage points, however, as 
such calculations can be misleading. The most sensible interpretation is that the 
percentages of students deemed proficient at both the basic and advanced levels 
increased noticeably during the first decade of the 21st century. 

Perhaps NCLB’s passage in 2002 dampened the prior rate of growth in the 
achievement of high-performing students. To ascertain whether that was the 
case we compared the rate of change in the NAEP math scores of the top 10 
percent of all 8th graders between 1990 and 2003 (before NCLB had begun to 
be implemented) with the rate of change after NCLB had become effective law. 
Between 1990 and 2003, the scores of the student at the 90th percentile rose 
from 307 to 321, an increment of 14 points, or a growth rate of 1.0 points a year. 
Between 2003 and 2009, the shift upwards for the 90th percentile was another 8 
points, or a change of 1.3 points a year. 27 

These findings are consistent with work by Thomas Dee and Brian Jacob 
(2009), who have undertaken a more complex analysis of the impact of NCLB on 
NAEP scores. In addition to estimating impacts on average performance across 
states, they estimate impacts on both very high and very low achieving students. 
Their study indicates that NCLB had positive impacts on the math performance 
of high- achieving students, even though larger impacts were observed for those 
at the bottom of the distribution. 28 

In short, the incapacity of American schools to bring students up to the 
highest level of accomplishment in mathematics is much more deep-seated than 
anything induced by recent federal legislation. 
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The Optimistic View from Prior Studies 

Our findings differ from two reports issued by Gary Phillips of the American 
Institutes of Research that compared the average performance in math of 8th-grade 
students in each of the 50 states with the average scores of 8th-grade students in 
other countries. 29 In his reports, Phillips relied on information from NAEP 2007 
and from math assessments in TIMSS: in his first report, achievement on TIMSS 
2003, and in his second report, achievement on TIMSS 2007. Phillips’ analysis 
compares average student achievement across countries, not the percentage of 
students performing at the advanced level, the focus of this report. His findings are 
distinctly more favorable to the U.S. than those shown by our analyses. While our 
study indicates that U.S. advanced student performance in math is tied for 31st 
place among countries surveyed, Phillips, in both his 2007 and his 2009 reports, 
finds U.S. students, on average, to be performing better than all but 8 countries. 

The opening sentences to the executive summary of his 2007 report draws 
quite buoyant conclusions: 

This report provides international benchmarks to help states see 
how students are doing in math within an international context. 

Good News — Most states are performing as well or better than 
most foreign countries. 

Bad News — The highest achieving states within the U.S. are still 
significantly below the highest achieving countries. 30 

Why do two studies that seem to be employing generally similar methodologies 
produce such strikingly different results? 

The answer to that puzzle is actually quite simple and has little to do with the fact 
that Phillips compares average student performance while our study focuses on the 
percentage of advanced students. The key difference is that the set of countries to 
which we compare the U.S. is noticeably different from the set of countries included 
in the Phillips comparisons. Many OECD countries, including those that had a high 
percentage of high-achieving students, participated in PISA 2006 (upon which our 
analysis is based) but did not participate in either TIMSS 2003 or TIMSS 2007 (the 
two surveys included in the Phillips studies). In fact, 16 countries that outscored the 
U.S. on the PISA 2006 test did not participate in TIMSS 2003 (see Appendix Tables 
C.l and C.2, p. 34-35). As a report by the U.S. National Center for Education Statistics 
has explained, “Differences in the set of counties that participate in an assessment can 
affect how well the U.S. appears to do internationally when results are released.” 31 




The set of countries 
to which we compare the 
U.S. is noticeably different 
from the set 
of countries included 
in the Phillips 
comparisons. 



30. Phillips (2007) p. 1. These findings 
received extensive media attention. As 
part of its favorable coverage, the New 
York Times quotes Thomas Toch, a former 
co-director of Education Sector as saying: 
“It shows we’re not doing as badly as some 
say.... We’re in the top half of the table, 
and a number of states are outperforming 
the majority of the nations in the study” 
(Dillon (2007)). 

31. Provasnik, Gonzales, and Miller 
(2009), p. 3. The report goes on to say: 
“One reason for this is that the average 
student performance in developed 
countries tends to be higher than in 
developing countries. As a result, the 
extent to which developing countries 
participate in an assessment can affect 
the international average of participating 
countries as well as the relative position of 
one country compared with the others.” 
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32. Hanushek and Woessmann (2008, 
2010). 


Put starkly, if one drops from a survey countries such as Canada, Denmark, 
Finland, France, Germany, and New Zealand, and includes instead such 
countries as Bulgaria, Botswana, Ghana, Iran, and Lebanon, the average 
international performance will drop, and the U.S. will look better relative to 
the countries with which it is being compared. (See Appendix C for a further 
discussion of the Phillips studies.) 

Discussion and Conclusions 

Math performance of young people in their adolescent years is shaped by 
a multiplicity of factors both within schools and outside of them. For that 
reason we do not identify in this report any single cause of the relatively 
small percentage of students in the U.S. who are performing at a high level 
of accomplishment. Sources of the problem may lie in the lack of initiative 
among students themselves, anti-educational pressures within the adolescent 
peer group culture, a lack of parental concern and support, anti-intellectual 
influences within the entertainment and mass media industries, a substantial 
minority population, high rates of in-migration, or even broader and deeper 
societal influences. But even though we suspect that one or more of these 
factors is at work, some of our findings point specifically to problematic 
elements within the nation’s schools. That even relatively advantaged groups 
in American society — white students and those with a parent who has a college 
education— do not generate a high percentage of students who achieve at the 
advanced level in math suggests, we submit, that schools are failing to teach 
students effectively. 

Raising the numbers of students performing at the highest level is not likely 
to be accomplished simply by allocating more dollars to our public schools. 

The U.S. is already among the world leaders in expenditures per pupil in K-12 
education, and the correlation between expenditures and achievement across 
OECD countries is virtually nil. 32 Spending more money on schools at a time of 
economic distress seems not only infeasible but also unlikely to produce the kind 
of changes that are needed if the U.S. is to remain among the highly productive 
countries of the world. 

This is not the place to identify the policy changes that might foster 
excellence. But we do note that policy initiatives have in recent years focused 
on the educational needs of low-performing students, a commendable target of 
policy. We see no sign that NCLB has been harmful to the highest-performing 
students. But we do fear that this policy environment leaves the impression that 
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there is no similar need to enhance the education of those students the STEM 
coalition has called “the best and brightest.” 

In its 2010 report, the “Rising above the Gathering Storm,” Committee 
of the National Academy of Sciences issued a second call for reform and 
innovation in math and science education. 33 “Our overall public school 
system... has shown little sign of improvement, particularly in mathematics 
and science,” it declared. Meanwhile, “many other nations have been markedly 
progressing, thereby affecting America’s relative ability to compete effectively 
for new factories, research laboratories, administrative centers— and jobs. 

While this progress by other nations is to be both encouraged and welcomed, 
so too is the notion that Americans wish to continue to be among those peoples 
who do prosper.” As its frontispiece, the report quotes Nobel Laureate Sir 
Ernest Rutherford: “Gentlemen, we have run out of money. It is time to start 
thinking.” We heartily agree. That is the idea that motivates this study. 



33. Members of the 2005 “Rising Above the 
Gathering Storm” Committee (2010) 



U.S. MATH PERFORMANCE IN GLOBAL PERSPECTIVE 25 



Appendix A 



APPENDIX A 

U.S. Science and Reading Performance 
in Comparative Perspective 

This report emphasizes math performance because that is the subject most 
closely correlated with increments in economic productivity and the subject for 
which common tests can be most readily designed for students coming from 
different language and cultural backgrounds. Two technical considerations 
have also dissuaded us from placing much emphasis on differences among 
nations in the percentage of students performing at the advanced level in 
science and reading. According to NAEP, only 3 percent of all U.S. students 
are said to have reached an advanced level of proficiency in these subjects. That 
should not be interpreted as showing that U.S. students are even more poorly 
taught in science and reading than in math. Rather, NAGB set the advanced 
proficiency standard in these two subjects at levels that were so high that only 3 
percent of U.S. students could attain them. Any standard set that high isolates 
such a small percentage of the population that it introduces the possibility of 
considerable error in measuring cross-country differences. Simply put, the tests 
may be subject to considerable error when viewed at a cutoff so far from the 
average performance. 

A second consideration arises with respect to the reading estimates. Because 
PISA was mal-administered within the U.S. in 2006, no PISA results are reported 
for that year. Thus, we cannot look directly at the Class of 2009 for reading, as 
we can in the case of math and science. Rather, we have to estimate results for 
that year by performing a similar analysis for the Class of 2006, with proper 
adjustments for the Class of 2009. That requires more assumptions than in the 
math and science analyses (see Appendix B for further methodological details). 
For those who nonetheless wish to make comparisons in these subject areas, 
Figures A.l and A.2 as well as Tables A.l, A.2 and A.3 present results for the 
science and reading performances of the Class of 2009. The reader is urged to 
peruse these results cautiously. The only conclusion we are willing to draw is that 
the U.S. trails many other countries in science and reading as well, although the 
lag is not as pronounced as in mathematics. 
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Table A.1 



Percentage of students in selected urban districts in the United States who are at 
the advanced level on NAEP 2005 

Percent advanced in... 



City Math Science Reading 



Atlanta 


1.3% 


(0.4) 


0.7% 


(0.3) 


1.0% 


(0.5) 


Austin 


8.9 


(1.2) 


3.9 


(0.6) 


3.1 


(0.8) 


Boston 


6.3 


(1.1) 


0.6 


(0.4) 


1.9 


(0.7) 


Charlotte 


8.6 


(0.9) 


N/A 


N/A 


3.2 


(0.5) 


Chicago 


1.7 


(0.6) 


0.7 


(0.3) 


1.1 


(0.3) 


Houston 


2.2 


(0.5) 


1.2 


(0.3) 


1.0 


(0.3) 


Los Angeles 


1.7 


(0.4) 


0.5 


(0.3) 


0.8 


(0.3) 


NYC 


4.5 


(1.1) 


1.6 


(0.4) 


1.4 


(0.4) 


San Diego 


4.0 


(0.7) 


1.4 


(0.4) 


2.0 


(0.7) 



All numbers are in percentage points. Standard errors in parentheses. 
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Class of 2009: Percentage of students at advanced level in science in U.S. states and countries 
participating in PISA 2006. (Figure A. 1 ) 



U.S. average 
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28 



educationnext.org 



hks.harvard.edu/pepg 



Appendix A 



Class of 2009: Percentage of students at advanced level in reading in U.S. states and countries 
participating in PISA 2006. (Figure A.2) 
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U.S. average 




Note: Excludes participating countries below 1%. 
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The U.S. trails 



many other countries in 
science and 
reading as well, 
although the lag is 
not as pronounced 
as in mathematics. 



Table A.2 



Percentage of students who are at the advanced level in all countries 
participating in PISA 2006 

Percent advanced in... 



Country Math Science Reading 



Argentina 


0.8% 


(0.3) 


0.1% 


(0.1) 


0.2% 


(0.1) 


Australia 


13.6 


(0.9) 


5.6 


(0.5) 


3.1 


(0.4) 


Austria 


13.0 


(1.1) 


3.0 


(0.4) 


2.7 


(0.4) 


Azerbaijan 


0.6 


(0.2) 


0.0 


(0.0) 


0.0 


(0.0) 


Belgium 


19.0 


(1.0) 


2.7 


(0.3) 


3.1 


(0.3) 


Brazil 


0.8 


(0.3) 


0.1 


(0.1) 


0.3 


(0.1) 


Bulgaria 


2.4 


(0.7) 


0.9 


(0.3) 


0.6 


(0.2) 


Canada 


14.7 


(1.0) 


5.1 


(0.4) 


5.0 


(0.5) 


Chile 


1.0 


(0.3) 


0.4 


(0.1) 


1.0 


(0.3) 


Colombia 


0.3 


(0.1) 


0.0 


(0.0) 


0.1 


(0.1) 


Croatia 


3.6 


(0.4) 


1.3 


(0.3) 


0.7 


(0.1) 


Czech Republic 


15.7 


(1.3) 


4.1 


(0.5) 


3.0 


(0.5) 


Denmark 


11.0 


(0.9) 


1.8 


(0.4) 


1.6 


(0.4) 


Estonia 


10.1 


(0.8) 


3.5 


(0.4) 


1.0 


(0.3) 


Finland 


20.4 


(1.2) 


8.5 


(0.6) 


5.3 


(0.6) 


France 


10.1 


(0.8) 


2.3 


(0.3) 


1.6 


(0.3) 


Germany 


13.1 


(1.0) 


4.0 


(0.4) 


3.2 


(0.4) 


Greece 


3.9 


(0.5) 


0.8 


(0.2) 


0.7 


(0.2) 


Hong Kong 


23.9 


(1.2) 


5.3 


(0.7) 


3.0 


(0.4) 


Hungary 


8.6 


(0.9) 


1.9 


(0.5) 


0.9 


(0.2) 


Iceland 


10.1 


(0.8) 


1.8 


(0.3) 


1.4 


(0.2) 


Indonesia 


0.2 


(0.2) 


0.0 


(0.0) 


0.0 


(0.0) 


Ireland 


7.9 


(0.7) 


2.9 


(0.4) 


3.7 


(0.5) 


Israel 


4.8 


(0.6) 


1.6 


(0.2) 


1.5 


(0.2) 


Italy 


4.9 


(0.5) 


1.2 


(0.2) 


1.3 


(0.2) 


Japan 


15.1 


(1.2) 


5.7 


(0.6) 


3.1 


(0.4) 


Jordan 


0.2 


(0.1) 


0.1 


(0.1) 


0.0 


(0.0) 


Korea 


23.2 


(1.7) 


3.0 


(0.6) 


7.6 


(1.1) 


Kyrgyzstan 


0.0 


(0.1) 


0.0 


(0.0) 


0.0 


(0.0) 


Latvia 


5.3 


(0.6) 


0.9 


(0.2) 


1.1 


(0.2) 


Liechtenstein 


16.1 


(1.8) 


4.5 


(1.2) 


3.5 


(1.3) 


Lithuania 


7.2 


(1.0) 


1.3 


(0.3) 


0.9 


(0.3) 


Luxembourg 


8.6 


(0.6) 


1.6 


(0.2) 


1.2 


(0.2) 


Macao 


14.1 


(0.7) 


1.0 


(0.2) 


0.5 


(0.1) 


Mexico 


0.6 


(0.2) 


0.0 


(0.0) 


0.1 


(0.0) 


Montenegro 


0.6 


(0.2) 


0.0 


(0.0) 


0.1 


(0.1) 


Netherlands 


17.6 


(1.1) 


4.4 


(0.5) 


2.2 


(0.5) 


New Zealand 


15.9 


(1.0) 


7.8 


(0.6) 


6.2 


(0.7) 


Norway 


8.3 


(0.6) 


1.6 


(0.3) 


2.3 


(0.3) 


Poland 


8.5 


(0.9) 


1.8 


(0.4) 


3.9 


(0.5) 


Portugal 


4.4 


(0.5) 


0.5 


(0.1) 


1.0 


(0.2) 


Qatar 


0.5 


(0.1) 


0.1 


(0.0) 


0.2 


(0.1) 


Romania 


0.9 


(0.3) 


0.0 


(0.0) 


0.1 


(0.1) 


Russia 


6.0 


(0.7) 


1.2 


(0.2) 


0.3 


(0.1) 


Serbia 


2.1 


(0.4) 


0.1 


(0.1) 


0.0 


(0.0) 


Slovakia 


8.9 


(1.0) 


1.5 


(0.3) 


1.4 


(0.3) 


Slovenia 


11.4 


(0.8) 


5.0 


(0.6) 


0.8 


(0.2) 


Spain 


5.7 


(0.5) 


1.1 


(0.2) 


0.3 


(0.1) 


Sweden 


10.0 


(0.8) 


2.6 


(0.4) 


3.4 


(0.4) 


Switzerland 


19.1 


(1.3) 


3.3 


(0.5) 


1.9 


(0.3) 


Taiwan 


28.0 


(1.5) 


4.4 


(0.5) 


0.9 


(0.2) 


Thailand 


1.0 


(0.2) 


0.0 


(0.0) 


0.0 


(0.0) 


Tunisia 


0.4 


(0.2) 


0.0 


(0.0) 


0.0 


(0.0) 


Turkey 


3.6 


(1.1) 


0.1 


(0.1) 


0.3 


(0.2) 


U.K. 


9.0 


(0.6) 


5.7 


(0.4) 


2.9 


(0.3) 


U.S. 


6.0 


(0.7) 


3.2 


(0.4) 


3.0 


(0.3) 


Uruguay 


2.4 


(0.4) 


0.3 


(0.1) 


0.9 


(0.2) 



All numbers are in percentage points. Standard errors in parentheses. U.S, results for Reading are 
from PISA 2003 as no 2006 results are available (See Appendix B) 
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Table A.3 



Percentage of students in U.S. states who are at the advanced level on NAEP 2005 

Percent advanced in... 



Math Science Reading 

All White Students of a parent All All 

State students students with College Education students students 



Alabama 


2.3% 


(0.6) 


3.6% 


(1.0) 


4.6% 


(1.4) 


1.4% 


(0.4) 


1.8% 


(0.6) 


Alaska 


5.8 


(0.6) 


8.5 


(0.9) 


N/A 


N/A 


N/A 


N/A 


N/A 


N/A 


Arizona 


4.6 


(0.4) 


7.6 


(0.9) 


9.6 


(1.0) 


1.8 


(0.4) 


1.6 


(0.3) 


Arkansas 


3.0 


(0.4) 


3.8 


(0.5) 


5.4 


(0.8) 


1.7 


(0.3) 


1.7 


(0.4) 


California 


4.5 


(0.4) 


7.2 


(0.7) 


9.5 


(0.9) 


1.8 


(0.2) 


1.6 


(0.2) 


Colorado 


6.3 


(0.8) 


8.4 


(1.0) 


11.0 


(1.4) 


4.4 


(0.6) 


3.5 


(0.5) 


Connecticut 


7.8 


(0.7) 


10.6 


(1.0) 


12.5 


(1.1) 


3.7 


(0.6) 


3.5 


(0.8) 


DC (DCPS) 


1.8 


(0.5) 


37.0 


(0.9) 


4.3 


(0.8) 


N/A 


N/A 


1.1 


(0.3) 


Delaware 


5.0 


(0.3) 


7.1 


(5.6) 


8.1 


(0.7) 


2.8 


(0.4) 


1.7 


(0.4) 


Florida 


4.6 


(0.7) 


7.1 


(1.0) 


8.0 


(1.1) 


2.0 


(0.3) 


2.0 


(0.3) 


Georgia 


4.3 


(0.5) 


6.3 


(0.9) 


7.9 


(1.1) 


2.8 


(0.4) 


2.5 


(0.5) 


Hawaii 


2.5 


(0.4) 


3.4 


(1.2) 


4.2 


(0.8) 


0.7 


(0.3) 


1.2 


(0.2) 


Idaho 


4.5 


(0.6) 


5.0 


(0.6) 


7.3 


(1.1) 


3.8 


(0.5) 


2.4 


(0.5) 


Illinois 


5.4 


(0.6) 


7.6 


(0.9) 


9.1 


(1.1) 


2.6 


(0.4) 


2.7 


(0.6) 


Indiana 


5.1 


(0.6) 


5.9 


(0.7) 


9.1 


(1.1) 


2.6 


(0.4) 


2.2 


(0.4) 


Iowa 


5.7 


(0.6) 


6.2 


(0.7) 


9.0 


(1.1) 


N/A 


N/A 


2.6 


(0.4) 


Kansas 


5.2 


(0.6) 


6.3 


(0.7) 


8.8 


(1.0) 


0.0 


(0.5) 


3.4 


(0.4) 


Kentucky 


3.4 


(0.5) 


3.6 


(0.6) 


6.6 


(1.1) 


3.0 


(0.3) 


3.0 


(0.5) 


Louisiana 


1.7 


(0.4) 


2.6 


(0.7) 


3.0 


(1.1) 


1.3 


(0.6) 


1.2 


(0.4) 


Maine 


5.0 


(0.5) 


5.1 


(0.5) 


7.4 


(0.9) 


2.8 


(0.6) 


3.5 


(0.5) 


Maryland 


6.8 


(0.7) 


9.8 


(1.1) 


11.6 


(1.2) 


3.6 


(0.6) 


3.6 


(0.5) 


Massachusetts 


11.4 


(0.8) 


12.6 


(1.0) 


17.1 


(1.3) 


5.8 


(0.6) 


5.2 


(0.7) 


Michigan 


5.5 


(0.8) 


6.8 


(1.2) 


8.5 


(1.5) 


4.1 


(0.6) 


2.0 


(0.5) 


Minnesota 


10.8 


(0.9) 


12.3 


(1.1) 


15.7 


(1.4) 


3.7 


(0.2) 


3.0 


(0.5) 


Mississippi 


1.3 


(0.3) 


2.4 


(0.7) 


2.2 


(0.7) 


0.7 


(0.5) 


0.8 


(0.3) 


Missouri 


4.1 


(0.5) 


5.0 


(0.7) 


6.9 


(1.0) 


3.1 


(0.5) 


2.6 


(0.4) 


Montana 


5.6 


(0.6) 


6.0 


(0.7) 


8.5 


(0.9) 


3.7 


(0.5) 


2.6 


(0.6) 


Nebraska 


6.0 


(0.6) 


6.8 


(0.7) 


9.0 


(0.8) 


N/A 


N/A 


2.6 


(0.4) 


Nevada 


3.1 


(0.5) 


4.6 


(0.8) 


6.2 


(1.2) 


1.2 


(0.2) 


1.4 


(0.5) 


New Hampshire 


6.5 


(0.7) 


6.5 


(0.7) 


10.2 


(1.2) 


4.2 


(0.9) 


3.9 


(0.5) 


New Jersey 


8.7 


(1.0) 


11.5 


(1.4) 


12.9 


(1.5) 


3.7 


(0.5) 


4.3 


(0.6) 


New Mexico 


1.4 


(0.3) 


3.2 


(0.7) 


2.9 


(0.8) 


1.2 


(0.3) 


1.0 


(0.2) 


New York 


6.3 


(0.5) 


7.7 


(0.8) 


10.2 


(0.8) 


N/A 


N/A 


3.3 


(0.5) 


North Carolina 


7.1 


(0.8) 


9.9 


(1.1) 


12.5 


(1.7) 


1.8 


(0.5) 


2.2 


(0.4) 


North Dakota 


4.8 


(0.5) 


5.3 


(0.6) 


6.4 


(0.9) 


3.9 


(0.5) 


2.9 


(0.6) 


Ohio 


6.6 


(0.6) 


7.6 


(0.7) 


11.0 


(1.1) 


4.0 


(0.6) 


3.7 


(0.7) 


Oklahoma 


2.4 


(0.4) 


3.0 


(0.5) 


4.3 


(0.7) 


2.1 


(0.5) 


1.1 


(0.4) 


Oregon 


7.3 


(0.8) 


7.9 


(0.9) 


12.7 


(1.3) 


3.4 


(0.5) 


2.6 


(0.5) 


Pennsylvania 


5.7 


(0.8) 


6.4 


(0.8) 


9.6 


(1.2) 


N/A 


N/A 


3.4 


(0.5) 


Rhode Island 


3.3 


(0.5) 


4.2 


(0.5) 


5.8 


(1.0) 


2.2 


(0.4) 


3.1 


(0.4) 


South Carolina 


6.7 


(0.7) 


10.8 


(1.2) 


11.3 


(1.2) 


2.1 


(0.4) 


1.9 


(0.4) 


South Dakota 


6.5 


(0.7) 


7.3 


(0.8) 


9.3 


(1.0) 


3.8 


(0.5) 


1.9 


(0.3) 


Tennessee 


2.9 


(0.4) 


3.7 


(0.5) 


5.8 


(0.9) 


2.5 


(0.5) 


1.4 


(0.4) 


Texas 


6.2 


(0.4) 


10.6 


(0.7) 


11.7 


(0.9) 


2.3 


(0.3) 


2.2 


(0.3) 


Utah 


4.7 


(0.6) 


5.4 


(0.6) 


7.5 


(0.8) 


3.5 


(0.5) 


2.3 


(0.4) 


Vermont 


8.8 


(0.7) 


9.0 


(0.7) 


14.0 


(1.1) 


4.3 


(0.5) 


4.1 


(0.4) 


Virginia 


7.9 


(0.9) 


10.3 


(1.3) 


13.0 


(1.5) 


4.4 


(0.6) 


3.3 


(0.5) 


Washington 


8.7 


(0.8) 


9.5 


(1.1) 


14.1 


(1.4) 


3.5 


(0.4) 


3.4 


(0.4) 


West Virginia 


1.4 


(0.3) 


1.3 


(0.3) 


3.0 


(0.6) 


1.8 


(0.4) 


1.4 


(0.3) 


Wisconsin 


6.7 


(0.7) 


7.6 


(0.8) 


10.8 


(1.2) 


4.7 


(0.5) 


3.4 


(0.5) 


Wyoming 


3.5 


(0.4) 


3.8 


(0.5) 


5.3 


(0.8) 


3.1 


(0.4) 


2.2 


(0.4) 



All numbers are in percentage points. Standard errors in parentheses. 
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APPENDIX B 

Methodology for Comparing U.S. States 
to International Performance 

The purpose of our analysis is to estimate the share of students in different 
countries that reach a competency level equivalent to the percentage of students 
in the U.S. who performed at the advanced level on NAEP 2005. 

For math, we start with the national share of 8th-grade U.S. students who 
reach the advanced level on NAEP 2005: 6.04 percent. One year later, this will 
roughly be the cohort of 15-year-olds who participated in PISA 2006. Both tests 
survey representative samples of the respective national student population. 

Thus, using the PISA 2006 microdata, we can calculate the PISA math test score 
at which the 93.96th percentile (100.00-6.04) of the U.S. student population 
performs. (All PISA calculations use the PISA sampling weights to yield 
nationally representative estimates.) We do this separately for each of the five 
plausible values of test performance provided by PISA 2006. Across the five 
plausible values, the PISA score at which the 93.96th percentile of U.S. students 
performs is on average 617.1 PISA points. 

Next, for each country participating in the PISA 2006 test, we calculate the 
share of students reaching this cut-off point from the PISA microdata. We do this 
separately for each plausible value and then take the average of the five estimates. 
This provides an estimate of the share of students in each PISA country who reach 
the level equivalent to the advanced level in 8th-grade math on NAEP 2005. The 
equivalent shares of students who reach the advanced level in 8th-grade math in 
each U.S. state are taken from NAEP 2005. The same applies for those cities that 
test representative samples of 8th graders on NAEP 2005. 

Issues with the U.S. PISA 2006 test in reading require a slightly expanded 
procedure. In the 2006 wave of the PISA test, no reading results are available 
for the U.S. because of an error in printing the test booklets. Some of the 
reading items had incorrect instructions, and as a consequence the mean 
performance in reading could not be accurately estimated. However, PISA 2003 
reading results for the U.S. are available, and equivalent NAEP proficiency 
estimates are available for U.S. 8th-graders in the reading component of NAEP 
2002. We are thus able to apply the proficiency shares from the NAEP 2002 
reading test to the U.S. performance on PISA 2003 in order to calculate the 
equivalent PISA cut-off scores in reading. Since both tests use the same scales 
in the two respective waves of the reading test — PISA 2003 and 2006 and NAEP 
2002 and 2005— this allows us to again calculate the shares of other countries' 
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students that reach these cut-off scores in PISA 2006 and compare them to the 
shares observed in the NAEP 2005 reading test. 

Some of the calculated differences in performance may simply reflect 
sampling uncertainty or measurement error. We therefore calculate whether 
the observed differences among states and countries are statistically significant 
(at the 5 percent level). The requisite standard errors are computed using 
the methodology described in Organization for Economic Co-operation and 
Development (2009), Chapters 7-9. These standard errors account for both 
sampling uncertainty (including the two-stage sampling design employed 
by PISA) and test unreliability (as captured by the five plausible values that 
represent the underlying probability distribution). NAEP 2007 standard errors 
are provided from the NAEP website (http://nces.ed.gov/nationsreportcard/, 
accessed August 28, 2010). Tests of significance were not calculated for urban 
districts or for reading and science test performances. 



APPENDIX C 

Further Reflections on the Phillips Studies 

Findings reported two years later by Phillips in his 2009 report do not differ 
materially from those he presented in 2007, as discussed in the main text of this 
report. However, in 2009 he introduced a grading system that gave the U.S. and 
most other countries mediocre grades on the familiar “A” to “F” scale. Although 
the grading aspect of the report received the most attention, 1 it is arbitrary. More 
important is the fact that Phillips still found U.S. 8th graders to be performing at 
a level “not significantly different from the OECD average” and above the level 
of a “broad cross-section of countries around the world.” The second report 
was less ebullient than the first, however, because the U.S. — and most other 
countries— received a grade of “C” or worse. 2 

Phillips also found that individual states do much better vis-a-vis other 
countries than we report. As can be seen in Figure 1, p.16, we find 44 of the 50 
states within the U.S. to be performing below 23 other countries. Meanwhile, 
Phillips reported that 21 U.S. states were scoring above the average for all 
OECD countries included in his survey. Moreover, every single state, including 
Mississippi, the lowest-performing, was scoring above the international average. 3 

U.S. is noticeably different from the set of countries included in the Phillips 
comparisons. Many OECD countries, including those that had a high percentage 
of high-achieving students, participated in PISA 2006 (upon which our analysis 



1. Cavanagh (2009) Finn (2009). 

2. Phillips (2009), p. 2. The tone of the 
2009 report differs from the 2007 report, 
because Phillips gives the United States a 
mediocre grade of “C,” a grade he gives to 
the average of all OECD countries. But no 
particular attention should be given to the 
specific grades Phillips gives, as all grading 
schemes are arbitrary. Those familiar with 
the rampant grade inflation in American 
higher education know how easily a “C+” 
grade can inflate to an “A-.” Phillips does the 
opposite by creating a highly deflationary 
grading scheme that gives a country an “A” 
only if the average score of its students is at 
or above the “advanced” level, which was set 
at the 94th percentile for all students taking 
the TIMSS test. This is clearly an extremely 
high standard, attained by no country in 
the world. Even Hong Kong, where the 
average student was more than one standard 
deviation above the assessment average, only 
received a B+. In our view, what counts is a 
country’s level of achievement, not its grade 
on some arbitrary scale. 

3. Phillips (2009), Table 4, pp. 24-25. 
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is based) but did not participate in either TIMSS 2003 or TIMSS 2007 (the two 
surveys included in the Phillips studies). In fact, 16 countries that outscored 




4. Provasnik, Gonzales, and Miller (2009), 
p. 3. The report goes on to say: “One 


the U.S. on the PISA 2006 test did not participate in TIMSS 2003 (see Table C.l 
and C.2). As a report by the U.S. National Center for Education Statistics has 


reason for this is that the average student 


explained, “Differences 










performance in developed countries tends 
to be higher than in developing countries. 


in the set of counties 


Table C.l 


As a result, the extent to which developing 
countries participate in an assessment 
can affect the international average of 


that participate in an 
assessment can affect how 


Countries scoring higher than the United States 
PISA 2006 and participation in TIMSS 


on 


participating countries as well as the relative 
position of one country compared with the 
others.” 

5. Both PISA and TIMSS use a scale that has 
a midpoint of 500 and a standard deviation 
of 100, which produces a range of nearly 


well the U.S. appears to 
do internationally when 
results are released.” 4 


Countries participating 
in PISA 2006 whose students 
had a higher average score 
than did the United States 


PISA Score 


Did not Did not 

participate participate 

in TIMSS in TIMSS 

2003 2007 


Other differences between 


Taiwan 


549 






1000. The midpoint and standard deviation 


our study and Phillips’s are 


Finland 


548 


• 


• 


on the PISA refer to student performance for 


Korea 


547 






those in OECD countries. On TIMSS they 


less significant. Our attention 


Hong Kong 


547 






refer to student performance for those in 


is focused on the percentage 
of students who are high 


Netherlands 


531 




• 


the 45 countries who participated in TIMSS 


Switzerland 


530 


• 


• 


in 1995, each country weighted equally. 


Canada 


527 


• 


• 


To link results across time, TIMSS links all 


achievers, while the focus 


Macao 


525 


• 


• 


tests taken since 1995 to results obtained in 


of the Phillips study is on 


Liechtenstein 


525 


• 


• 


that year. In 1995 the average for 7th- and 
8th-grade students on the TIMSS was 500 
points, with a standard deviation 100 points. 
That TIMSS norm was based upon results 
from all countries who participated in TIMSS 
1995, including 27 members of the OECD 
(where Scotland and England are considered 
separately), and 18 countries that were not 
members of the OECD, many of them from 
the developing world. The non-OECD 
countries that participated in the 1995 


Japan 


523 






average student performance. 


New Zealand 


522 




• 


But even if we shift our focus 


Australia 


520 






Belgium 


520 




• 


to differences in average 


Estonia 


515 




• 


performance, the two studies 
yield dramatically different 


Denmark 


513 


• 


• 


Czech Republic 


510 


• 




Iceland 


506 


• 


• 


findings. For example, the 
average U.S. score of 474 


Austria 


505 


• 


• 


Germany 


504 


• 


• 


TIMSS were Argentina, Bulgaria, Colombia, 


Slovenia 


504 






Cyprus, Hong Kong, Indonesia, Iran, Israel, 


points on PISA 2006 falls well 


Sweden 


502 






Kuwait, Latvia, Lithuania, Philippines, 


short of the OECD average 


Ireland 


501 


• 


• 


Romania, Russia, Singapore, Slovenia, 


France 


496 


• 


• 


South Africa, and Thailand. Because of 


of 500 points on this test. 5 


Poland 


495 


• 


• 


this different norming of the tests, equally 


But on the TIMSS 2007, the 
U.S. average is 508 points, 


U.K.* 


495 






performing students will have higher scores 


Slovakia 


492 




• 


on TIMSS tests than PISA tests, and so it is 


Hungary 


491 






not surprising that the United States average 


a score almost equivalent 
to the OECD average of 


Luxembourg 


490 


• 


• 


on the TIMSS 2007 was 508 points, while its 


Norway 


490 






average on PISA 2006 was 474. Many OECD 
countries decided not to participate in the 
TIMSS 2003 and TIMSS 2007, while other 
countries have joined, so the international 


Latvia 


486 




• 


511 points. The OECD 


Lithuania 


486 






averages on the TIMSS are 


Spain 


480 


• 


• 


Russia 


476 






average fell from 500 points in 1995 to 466 
points. Obviously, that should not be taken 
as evidence that students worldwide are 


misleading, however, as they 


Azerbaijan 


476 


• 


• 


are based on results from 


U.S. 


474 






performing at a lower level, but rather that 
the composition of countries participating in 
the TIMSS has changed over the years. 


just the 11 OECD countries 
that participated in TIMSS 


♦Scotland and England participated separately in TIMSS 2003 and 2007. Wales and 
Northern Ireland did not participate. 
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Table C.2 



Countries participating 
in TIMSS but not 



2007. As stated above, the other 19 OECD countries, many of them the high-scoring 
countries, did not participate in this assessment. Philips also compares states to an 
international average that includes the scores from all 48 countries that participated 
in TIMSS 2007. That average of 461 is well below the U.S. score, but, of course, it 
includes many developing countries. 6 In contrast, the 
official international average for PISA is based strictly on the 
average for all countries that are members of the OECD. 

In his 2009 study, Phillips indicates a preference for 
the TIMSS 2007 over PISA 2006, because TIMSS 2007 
was administered to 8th graders in the same year as NAEP 
2007, while PISA 2006 was administered to 15-year-olds 
one year after NAEP 2005 was administered to 8th graders. 
Theoretically, the administration of the two tests to the same 
grade levels in the same year is an advantage when making 
international comparisons. But, practically speaking, that 
advantage is relatively minor. Phillips himself gets much the 
same results regardless of whether he compares NAEP 2007 
results to TIMSS 2003 (as he did in his 2007 report) or to 
TIMSS 2007 (as he did in his 2009 report). 

Finally, Phillips suggests that PISA 2006 tests math 
“literacy” while TIMSS 2007 assesses math “proficiency” 
with a test that is more closely aligned to the curriculum 
offered by the U.S. But just as the words literacy and 
proficiency are virtually inter-changeable, the two tests 
are more alike than they are different. As Phillips himself 
demonstrates, the correlation between average student 
performances across countries on the PISA 2006 and 
TIMSS 2007 is 0.93. 7 When two indicators of student 
performance yield such similar results, one generally 
assumes them to be different measures of the same thing. 
Random differences in sampling and item construction 
can easily account for any observed differences in results. 

In sum, the major difference between this study 
and the Phillips reports is the countries with which the 
U.S. is being compared. We include in our comparison 
all countries of the OECD, many of them among the 
highest- achieving countries, while Phillips has included 
in his comparisons only 1 1 of those countries. 



PISA 2006 




TIMSS 2003 




Participants with 


TIMSS 


a higher score 


2003 


than the U.S. 


Score 


Singapore 


605 


Korea 


589 


Hong Kong 


586 


Taiwan 


585 


Japan 


570 


Belgium-Flemish 


537 


Netherlands 


536 


Estonia 


531 


Hungary 


529 


Malaysia 


508 


Latvia 


508 


Russia 


508 


Slovakia 


508 


Australia 


505 


U.S. 


504 


TIMSS 2007 




Participants with 


TIMSS 


a higher score 


2007 


than the U.S. 


Score 


Taiwan 


598 


Korea 


597 


Singapore 


593 


Hong Kong 


572 


Japan 


570 


Hungary 


517 


England 


513 


Russia 


512 


U.S. 


508 



Note: Countries in italic did not 
participate in PISA 2006. 



6 . Phillips (2009), p. 18. 

7. Phillips (2009), p. 36. 
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